Image and video coding techniques use different color formats to represent dolor information of picture elements (pixels or samples). The color information may be expressed differently depending on the use case. For example, it is common for displays to have three color elements, Red (R), Green (G) and Blue (B), which are lit at different intensities depending on what color is to be displayed. It becomes therefore convenient to send the pixel information in RGB color format to the display. Since the signal is digital the intensity of each component of the pixel must be represented with a fixed number of bits, referred to as the bit depth of the component. A bit depth of n can represent 2n different values, i.e. 256 values per component for 8 bits and 1024 values per component for 10 bits.
The YUV color format describes a color by using the luminance (luma) color component Y and two chrominance (chroma) color components U and V (or Cb and Cr respectively). The luminance color component (Y) represents the brightness whereas the chrominance components (U and V) contain the color differences. Representation of a pixel in YUV (YCbCr) format is convenient for video coding. Namely, the human visual system (HVS) is more sensitive to luminance than to chrominance, meaning that luminance can be represented with higher accuracy than chrominance.
Since the HVS is not equally sensitive to all levels of luminance in linear light intensity the video is usually transferred into a perceptual domain Y′UV (Y′CbCr) where Y′ is referred to as luma. The conversion between RGB and Y′UV for HDTV is to be made using the following matrix multiplications:
      [                                        Y            ′                                                U                                      V                      ]    =                              [                                                    0.2126                                            0.7152                                            0.0722                                                                                      -                  0.09991                                                                              -                  0.33609                                                            0.436                                                                    0.615                                                              -                  0.55861                                                                              -                  0.05639                                                              ]                ⁡                  [                                                    R                                                                    G                                                                    B                                              ]                    ⁢                          [                                    R                                                G                                                B                              ]        =                  [                                            1                                      0                                      1.28033                                                          1                                                      -                0.21482                                                                    -                0.38059                                                                        1                                      2.12798                                      0                                      ]            ⁡              [                                                            Y                ′                                                                        U                                                          V                                      ]            
Fourcc.org holds a list of defined YUV and RGB formats. The most commonly used pixel format for standardized video codecs (e.g. H.264/AVC, MPEG-4, HEVC) is Y′UV 4:2:0 (aka YV12 or Y′CbCr 4:2:0) planar where the U and V chrominance components are subsampled in both vertical and horizontal direction and the Y′, U and V components are stored in separate chunks for each picture. Thus for a bit depth of 8 bits per component, the number of bits per pixel is 12 where 8 bits represents the luminance and 4 bits the two chrominance components. A pixel format that does not have sub-sampling in any of its components is referred to have a 4:4:4 sampling format, e.g. Y′CbCr 4:4:4 or RGB 4:4:4.
The High Efficiency Video Coding (HEVC), also known as H.265 has a Cross-Component Prediction (CCP) tool for predicting the residuals of the chrominance blocks of samples (pixels) from the residuals of the luminance blocks of samples (pixels). The tool was initially proposed during the development of H.265/HEVC RExt that supports higher bit depths and 4:2:2 and 4:4:4 chroma sampling formats for the HEVC.
The residual of a chroma component is calculated as:rCR={circumflex over (r)}CR−α·{circumflex over (r)}CM where {circumflex over (r)}CM is the luminance component residual, {circumflex over (r)}CR is the residual of the remaining component at the same spatial location and α is a weighting factor. The α parameter is signaled at the block level in the bitstream for each of the two chroma components.
Chroma Intra Prediction using Linear Model (LM) is another tool proposed during the development of HEVC. With this tool the samples of a chroma block are predicted from the reconstructed downsampled luma samples by using a linear model as follows:predC(i,j)=α·recL(i,j)+βwhere predC(i, j) is the chroma sample to be predicted, recL(i, j) is the corresponding downsampled reconstructed luma sample, and α and β are the model parameters. The values of α and β are derived from the reconstructed neighboring samples of the current block. The normalized covariance between luma and chroma reference samples is calculated and taken as α. The offset β is obtained by fitting the linear model to the reference samples, utilizing the function above given the known α.
Contribution ITU-T COM16C 806E investigates a number of tools that show large improvements over HEVC. This contribution later becomes the KTA test model for the initial development of the next generation video codec after HEVC. One of the tools in KTA is the Chroma Intra Prediction using Linear Model (LM), with the addition that the residual of one of the chroma components may also be predicted from the residual of the other chroma component according to:predCr*(i,j)=predCr(i,j)+α·resiCb′(i,j)